Earphones with activity controlled output

ABSTRACT

The present disclosure relates to an earphone apparatus and method. The earphone apparatus includes an earpiece, including a speaker, configured for arrangement relative to a user&#39;s ear for listening to audio from the speaker, a bio-sensor, a motion sensor, and a controller configured to determine an activity state of the user based on aggregated sensor data from the bio-sensor and the motion sensor over time, and to control audio output to the user based on the determined activity state.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. patent application Ser. No.15/597,717, filed May 17, 2017, which is a continuation of InternationalApplication No. PCT/SE2015/051220 filed on Nov. 16, 2015, which claimsthe benefit under 35 USC § 119(a) of Swedish Patent Application No.1451410-3, filed on Nov. 21, 2014, in the Swedish Patent andRegistration Office, the entire disclosures of all of which areincorporated herein by reference for all purposes.

BACKGROUND 1. Field

The present application generally relates to the field of earphones, andin particular, earphones with an integrated sensor to control audiooutput.

2. Description of Related Art

Generally, within the technical field of earphones a user selects whathe/she wants to listen to. It is common that users fall asleep whenlistening to music or other audio. In state of the art music players itis possible to set a timer for how long the music is going to playbefore stopping. However, this function requires that the user knowswhen he/she will fall asleep, or how soon the user expects to fallasleep. In many cases the user does not know how long time it will taketo fall asleep; 5 minutes or 50 minutes, and therefore correctly settinga timer is not an easy task.

SUMMARY

In one general aspect, an earphone apparatus includes an earpiece,including a speaker, configured for arrangement relative to a user's earfor listening to audio from the speaker, a bio-sensor, a motion sensor,and a controller configured to determine an activity state of the userbased on aggregated sensor data from the bio-sensor and the motionsensor over time, and to control audio output to the user based on thedetermined activity state.

The controller may be configured to determine the activity state of theuser based on learned activity states of the user from the aggregatedsensor data over time.

To determine the activity state of the user, the controller maydetermine a preliminary activity state of the user based on the sensordata from the bio-sensor, and confirm the preliminary activity statebased on the sensor data from the motion sensor, to determine theactivity state of the user.

The apparatus may further include a microphone, wherein the controllermay be further configured to control the microphone to capture ambientsounds, and, to determine the activity state of the user, the controllermay determine a preliminary activity state of the user based on thesensor data from the bio-sensor and the sensor data from the motionsensor, and confirm the preliminary activity state based on adetermination whether the preliminary activity state of the usercorrelates to an activity state represented by the captured ambientsounds.

The bio-sensor and the controller may be configured within the earpiece,and the microphone may be connected to the earpiece or configured withinthe earpiece.

When the sensor data from the motion sensor and the sensor data from thebio-sensor indicate the user is resting, the controller may determinewhether the activity state is a sleeping state trained based onaggregated detected movements over time by the motion sensor that arebelow a predetermined threshold, the trained sleeping state beingupdated over time.

The controller may determine where the sensor data of the motion sensorand the sensor data of the bio-sensor falls on a determinedsleep-to-workout scale based on detected changes in movements of theuser, to determine the activity state of the user.

The controller may be configured to determine between plural associatedsub-states of the determined activity state based on detected changes inheart rate within a range of heart rates indicated by the sensor data ofthe bio-sensor and corresponding to the determined activity state, andselectively control the audio output based on the determination betweenthe plural sub-states.

When the determined activity state is a workout state, the pluralassociated sub-states may include two or more of warming up, running,lifting weights, bicycling, climbing, dancing, jumping, doing yoga, andstretching sub-states.

The controller may consider between the plural associated sub-states ofthe determined activity state based on location information regarding alocation of the user.

The controller and the bio-sensor may be configured within the earpieceand the controller may determine the location of the user.

The determination between the plural sub-states may be based on adetermined progression between the plural sub-states.

The earpiece may be an in-ear earpiece or an out of ear earpiece, andthe controller and at least one of the motion sensor and the bio-sensormay be configured within the earpiece.

The motion sensor and the bio-sensor may be configured in the earpiece.

The apparatus may further include a remote control unit including asecond motion sensor, where the remote control unit is separate from theearpiece, and the controller may determine the activity state furtherbased on sensor data from the second motion sensor.

The earpiece may be an in-ear earpiece or an out of ear earpiece, andthe apparatus may further include another in-ear or out of ear earpiecethat includes a corresponding speaker, the controller may be configuredin the earpiece or the other earpiece, and the motion sensor and thebio-sensor may be both configured within one of the earpiece and theother earpiece or the motion sensor configured in the earpiece and thebio-sensor configured in the other earpiece, and the earpiece and/or theother earpiece may include a power harvesting component to perform atrickle energy capture to power the motion sensor and the bio-sensor.

The motion sensor may be an acceleration sensor and the bio-sensor is aheart rate sensor.

The earpiece may further include a communication module and wirelesslyconnects with a media player to obtain the audio output using thecommunication module.

The controller may be further configured to transmit control data to themedia player, based on the determined activity state, for thecontrolling of the audio output based on the determined activity state.

The power harvesting component may harvest power from a wiredconnection, to the earpiece and/or the other earpiece, that providesaudio to the earpiece and/or the other earpiece from a media player.

The controller may be further configured to selectively controlrespective periods of time the motion sensor and the bio-sensor arecontrolled to respectively measure motion and bio-signals, and the powerharvesting component may selectively perform between the trickle energycapturing, with a storage of captured energy in a local power storage,during power off times of the bio-sensor of the respective periods oftime and a supplying of the stored captured energy from the local powerstorage to the bio-sensor during power on times of the bio-sensor of therespective periods of time.

The motion sensor may be an acceleration sensor and the bio-sensor is aheart rate sensor, the earpiece may be an in-ear earpiece or an out ofear earpiece, and the apparatus may further include another in-ear orout of ear earpiece that that includes a corresponding speaker, with themotion sensor and the bio-sensor being both configured within one of theearpiece and the other earpiece or the motion sensor being configured inthe earpiece and the bio-sensor being configured in the other earpiece.

The apparatus may further include a remote control unit including asecond motion sensor, the remote control unit may be separate from theearpiece and the other earpiece, and the controller may determine theactivity state further based on sensor data from the second motionsensor.

The controller may be further configured to filter the sensor data fromthe bio-sensor using the sensor data from the motion sensor and/or thesensor data from the second motion sensor.

The controller may be further configured to monitor the sensor data ofthe motion sensor and/or the sensor data from the second motion sensorfor detected indications that the user is inputting a touch or tapcommand, and to not rely on, for the determining of the activity stateof the user, the sensor data of the motion sensor and/or the sensor datafrom the second motion sensor that is determined to correspond to thedetected indications that the user is inputting the touch or tapcommand.

The controller may be configured to determine whether the touch or tapis a command to control a pausing or resuming of audio output, tocontrol a change in a song, genre, or playlist in the audio output, orto control an activation of a voice control.

The controller may be configured in the remote control unit, and theearpiece, the other earpiece, and the remote control unit are wiredlyconnected to each other.

The remote control unit may further include a power harvesting componentto perform a trickle energy capture to power the motion sensor and thebio-sensor, the cable may be connected to a media player that providesthe audio output, the controller may be further configured toselectively control respective periods of time the motion sensor and thebio-sensor are controlled to respectively measure motion andbio-signals, and the power harvesting component may selectively operatebetween the trickle energy capturing, with a storage of captured energyin a local power storage, during power off times of the bio-sensor ofthe respective periods of time and a supplying of the stored capturedenergy from the local power storage to the bio-sensor during power ontimes of the bio-sensor of the respective periods of time.

The controller and the bio-sensor may be configured within the earpiece,and the controller may be further configured to monitor the sensor dataof the motion sensor for detected indications that the user is inputtinga touch or tap command, and to not rely on, for the determining of theactivity state of the user, the sensor data of the motion sensor that isdetermined to corresponds to the detected indications that the user isinputting the touch or tap command.

The motion sensor may be configured in the earpiece or another otherearpiece of the earphone apparatus.

The controller may be configured to determine whether the touch or tapis a command to control a pausing or resuming of audio output, tocontrol a change in a song, genre, or playlist in the audio output, orto control an activation of a voice control.

The earpiece may include an adjustable loop formed by a first attachmentpoint of a cable and the earpiece and a second attachment point of thecable and the earpiece, the first attachment point and the secondattachment point being configured so the adjustable loop selectivelyinteracts with a concha of the user's ear to fixedly position theearpiece relative to the ear, and the cable may wiredly provide theaudio output to the earpiece.

The earpiece may be an in-ear earpiece or an out of ear earpiece, themotion sensor and the bio-sensor may be configured within the earpiece,and the earpiece may include an adjustable loop formed by a firstattachment point of a cable and the earpiece and a second attachmentpoint of the cable and the earpiece.

The earpiece may be an in-ear earpiece or an out of ear earpiece, theapparatus may further include another in-ear or out of ear earpiece, themotion sensor and the bio-sensor may be both configured within one ofthe earpiece and the other earpiece or the motion sensor configured inthe earpiece and the bio-sensor is configured in the other earpiece, andthe earpiece may include an adjustable loop formed by a first attachmentpoint of a cable and the earpiece and a second attachment point of thecable and the earpiece.

The cable may wiredly connect the earpiece to the other earpiece.

The controller may be further configured to determine an activity statetransition between activity states of the user based on both the sensordata from the bio-sensor and the sensor data from the motion sensor todetermine the activity state of the user.

The controller may be further configured to determine a preliminaryactivity state of the user based on the sensor data from the bio-sensor,to confirm the preliminary activity state based on the sensor data fromthe motion sensor to determine the activity state of the user.

To control the audio output, the controller may adjust the audio outputto match a determined low activity when the determined activity state isa predetermined low activity state and adjust the audio output to matcha determined high activity when the determined activity state is apredetermined high activity state.

The controlling of the audio output may include one of stopping,pausing, playing, increasing volume, decreasing volume, matching a beatper minute, a cadence, rhythm, predetermined user selection, genre, andpredefined user playlist or specific track selection.

The controller may be further configured to obtain cadence sensor datafrom an external exercise equipment cadence sensor, and to determine theactivity state further based on the obtained cadence sensor data.

The controller may be further configured to determine the activity statefrom among plural user preset activity states set through a userinterface connected to the controller.

In one general aspect, an earphone method of an earphone apparatushaving earpieces configured for respective arrangements relative to auser's ears for listening to audio includes controlling respectivemeasuring of bio-signals and motion changes of a user by the earpieces,aggregating sensor data of the measured bio-signals and motion changesof the user over time, determine an activity state of the user based onthe aggregated sensor data, and selectively controlling the audio outputto the user based on the determined activity state.

The method may further include learning activity states of the user fromthe aggregated sensor data, and the determining the activity state maydetermine the activity state from among the learned activity states.

The learning of the activity states may include learning a sleepingstate based on aggregated detected movements over time of the user thatare below a predetermined threshold, and updating the learned sleepingstate over time.

The determining of the activity state may include determining betweenplural associated sub-states of the determined activity state based ondetected changes in heart rate within a range of heart rates indicatedby the measured bio-signals and corresponding to the determined activitystate, and the selective control of the audio output may be performedbased on the determination between the plural sub-states.

Then the determined activity state is a workout state, the pluralassociated sub-states may include two or more of warming up, running,lifting weights, bicycling, climbing, dancing, jumping, doing yoga, andstretching sub-states.

The method may further include determining a location of the user, andthe determining between the plural associated sub-states may includedetermining between the plural associated sub-states of the determinedactivity state based on the determined location of the user.

The determination between the plural sub-states may be based on adetermined progression between the plural sub-states.

The determining of the activity state may be further based on sensordata from a motion sensor of a remote control unit separate from theearpieces.

The method may further include monitoring the measured motion changesfor detected indications that the user is inputting a touch or tapcommand, and the determining of the activity state of the user mayinclude not relying on, for the determining of the activity state of theuser, the measured motion changes that are determined to correspond tothe detected indications that the user is inputting the touch or tapcommand.

The monitoring of the measured motion changes may include determiningwhether the touch or tap is a command to control a pausing or resumingof the audio output, to control a change in a song, genre, or playlistin the audio output, or to control an activation of a voice control.

The method may further include performing a trickle energy capturing ofenergy to power a motion sensor and bio-sensor of the earpieces, whichrespectively perform the measuring of the bio-signals and the motionchanges

The trickle energy capturing of energy may include capturing trickleenergy from a cable connected to at least one of the earpieces, thecable being connected to a media player that is separate from theearpieces and which provides the audio output.

The method may further include selectively controlling respectiveperiods of time the motion sensor and the bio-sensor are controlled torespectively measure the motion changes and the bio-signals, andselectively performing between the trickle energy capturing, with astorage of captured energy in a local power storage, during power offtimes of the bio-sensor of the respective periods of time and asupplying of the stored captured energy from the local power storage tothe bio-sensor during power on times of the bio-sensor of the respectiveperiods of time.

The controlling of the audio output may be performed by transmittingcontrol data by a controller included in one of the earpieces to a mediaplayer to control a providing of the audio output, the controllerperforming the determining of the activity state of the user.

The controlling of the audio output may include one of stopping,pausing, playing, increasing volume, decreasing volume, matching a beatper minute, a cadence, rhythm, predetermined user selection, genre, andpredefined user playlist or specific track selection.

The method may further include obtaining cadence sensor data from anexternal exercise equipment cadence sensor, and the determining of theactivity state may include determining the activity state further basedon the obtained cadence sensor data and/or the controlling of the audiooutput is further based on the obtained cadence sensor data.

The method may further include providing a user interface to collectuser defined preset activity state information, and the determining ofthe activity state may include determining the activity state from amongplural preset activity states that are defined based on the presetactivity state information.

In one general aspect, provided is a non-transitory computer readablemedium storing instructions, which when implemented by a processor,cause the processor to implement one or more or all operations describedherein.

In one general aspect, an earphone apparatus includes an earpiece,including a speaker, configured for arrangement relative to a user's earfor listening to audio from the speaker, a bio-sensor, a motion sensor,and a controller configured to determine an activity state transitionbetween activity states of the user based on both sensor data from thebio-sensor and sensor data from the motion sensor, and to control audiooutput to the user based on the determined activity state transition.

When the activity state transition is a falling asleep state, as atransition between a resting state to a sleep state, the controller maycontrol the audio output to the user to gradually decrease a volume ofaudio while the user is in the falling asleep state.

When the controller determines that an activity state of the user is asleep state based on both the sensor data from the bio-sensor and thesensor data from the motion sensor, the controller may control the audiooutput to the user to cease.

The earpiece may be an in-ear earpiece or an out of ear earpiece, andthe controller and at least one of the motion sensor and the bio-sensormay be configured within the earpiece.

The motion sensor and the bio-sensor may be configured in the earpiece.

The apparatus may further include a remote control unit including asecond motion sensor, where the remote control unit may be separate fromthe earpiece, and the controller may determine the activity statetransition further based on sensor data from the second motion sensor.

The earpiece may be an in-ear earpiece or an out of ear earpiece, andthe apparatus may further include another in-ear or out of ear earpiece,and motion sensor and the bio-sensor may both be configured within oneof the earpiece and the other earpiece or the motion sensor may beconfigured in the earpiece and the bio-sensor configured in the otherearpiece.

The apparatus may further include a remote control unit including asecond motion sensor, the remote control unit may be separate from theearpiece and the other earpiece and the controller may determine theactivity state transition further based on sensor data from the secondmotion sensor.

The controller may be configured in the remote control unit.

The earpiece, the other earpiece, and the remote control unit may bewiredly connected through a same cable.

In one general aspect, an earphone apparatus includes an earpiece,including a speaker, configured for arrangement relative to a user's earfor listening to audio from the speaker, a bio-sensor, a motion sensor,and a controller configured to determine a preliminary activity state ofthe user based on the sensor data from the bio-sensor, to confirm thepreliminary activity state based on the sensor data from the motionsensor to determine an activity state of the user, and to control audiooutput to the user based on the determined activity state.

Other features and aspects will be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a side view of an earphone according to one or moreembodiments;

FIG. 2 illustrates a front view of an earphone according to one or moreembodiments;

FIGS. 3-4 illustrate a front view of an earphone in use according to oneor more embodiments;

FIG. 5 illustrates a perspective view of an earphone according to one ormore embodiments;

FIG. 6 illustrates an earphone system according to one or moreembodiments;

FIG. 7 illustrates a flow chart describing an earphone method accordingto one or more embodiments; and

FIG. 8 illustrates an earphone system according to one or moreembodiments.

Throughout the drawings and the detailed description, unless otherwisedescribed or provided, the same drawing reference numerals will beunderstood to refer to the same or like elements, features, andstructures. The drawings may not be to scale, and the relative size,proportions, and depiction of elements in the drawings may beexaggerated for clarity, illustration, and convenience.

DETAILED DESCRIPTION

The following detailed description is provided to assist the reader ingaining a comprehensive understanding of the methods, apparatuses,and/or systems described herein. However, various changes,modifications, and equivalents of the methods, apparatuses, and/orsystems described herein will be apparent after an understanding of thedisclosure of this application. For example, the sequences of operationsdescribed herein are merely examples, and are not limited to those setforth herein, but may be changed as will be apparent after anunderstanding of the disclosure of this application, with the exceptionof operations necessarily occurring in a certain order. Also,descriptions of features that are known in the art may be omitted forincreased clarity and conciseness.

The features described herein may be embodied in different forms, andare not to be construed as being limited to the examples describedherein. Rather, the examples described herein have been provided merelyto illustrate some of the many possible ways of implementing themethods, apparatuses, and/or systems described herein that will beapparent after an understanding of the disclosure of this application.

The terminology used herein is for the purpose of describing particularexamples only, and is not to be used to limit the disclosure. As usedherein, the singular forms “a,” “an,” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. As used herein, the term “and/or” includes any one and anycombination of any two or more of the associated listed items. As usedherein, the terms “include,” “comprise,” and “have” specify the presenceof stated features, numbers, operations, elements, components, and/orcombinations thereof in at least one embodiment, such as when it isindicated that such stated features, numbers, operations, elements,components, and/or combinations thereof may be included in an example.However, the use of the terms “include,” “comprise,” and “have” in theSpecification do not preclude the presence or addition of one or moreother features, numbers, operations, elements, components, and/orcombinations thereof in other embodiments, and do not preclude in theSpecification the lack of presence of any of such features, numbers,operations, elements, components, and/or combinations thereof in stillother embodiments unless explicitly or contextually/implicitly clearlyexplained otherwise.

In addition, terms such as first, second, A, B, (a), (b), and the likemay be used herein to describe components. Each of these terminologiesis not used to define an essence, order, or sequence of a correspondingcomponent but used merely to distinguish the corresponding componentfrom other component(s).

Throughout the specification, when an element, such as a layer, region,or substrate, is described as being “on,” “connected to,” or “coupledto” another element, it may be directly “on,” “connected to,” or“coupled to” the other element, or there may be one or more otherelements intervening therebetween. In contrast, when an element isdescribed as being “directly on,” “directly connected to,” or “directlycoupled to” another element, there can be no other elements interveningtherebetween. Likewise, expressions, for example, “between” and“immediately between” and “adjacent to” and “immediately adjacent to”may also be construed as described in the foregoing.

Unless otherwise defined, all terms, including technical and scientificterms, used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this disclosure pertainsconsistent with and after an understanding of the present disclosure.Terms, such as those defined in commonly used dictionaries, are to beinterpreted as having a meaning that is consistent with their meaning inthe context of the relevant art and the present disclosure, and are notto be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

A function or an operation illustrated in a block may be performed notin a sequential order according to examples. For example, functions oroperations illustrated in successive blocks may be actually performedconcurrently, or an order of the blocks may be changed based on relatedfunctions or operations.

As described above, in the field of earphones, a user may set a sleepmode, which may control when audio output is scheduled to cease.However, the user will not know when sleep will begin, and often may noteven set the sleep mode or may forget to set the sleep mode.Consequently, the user may set the timer with a too long time span, ormay not activate the sleep timer at all, and may therefore accidently bewoken from the audio output and/or waste battery resources of the playerapparatus and the earphones while asleep.

Accordingly, it has been found that there is a desire for an improvedearphone technology with an improved sleep function, as only an example.It has been found that a state of the user may be determined with higheraccuracy when both of two different types of sensor data are consideredto determine the state of the user, compared to previous approacheswhere a separate media player may determine what activity the user iscurrently performing based on sensed motion information or separatelywhere a media player determines an activity level based on a sensedheart rate from a heart rate monitor.

For example, by considering both data from an example pulse ratedetector and a motion sensor included in earphones worn by the user, thedetermination of a state of the user can be performed more accuratelythan previous approaches, as data from only one of the pulse ratedetector and the motion sensor may be insufficient to determine a stateof the user, e.g., for providing an appropriate and correct audiooutput. For example, a user's pulse may increase due to a state ofnervousness or stress. However, if a controller of the earphone systemdetermines the “workout-state” of the user based on the high pulse rate,and thereby selectively controls an increased tempo audio output, theaudio output may not be desirable to the user since the fast beat musicmay only exacerbate the stress state of the user. Rather, by consideringboth data from the example pulse rate detector and the example motionsensor, a controller of an earphone system may correctly determine thata user has a high pulse that is not coincident with activity by the useror not generated by such user activity, and thus, may determine that theuser is in a stressed state. In such an example, the controller maythen, based on the determined stressed state, control or adjust audiooutput to be a calming or calmer lower speed beat relaxing music orpreset relaxing sounds. This may be desirable for persons inpsychological distress, for example.

As another example, earphones may be provided with an improved sleepfunction. In such an example, audio output may be controlled to bedependent on a user's determined state. For example, audio output may becontrolled dependent on the user's determined state and correspondingpredetermined and/or personalized audio output. The determinable statesof the earphone system may be set by the user and/or be predetermined.Also, associated predetermined and/or personalized audio outputoperations may then be controlled based on the determined user's state.For example, select audio output operations may be differently performeddependent on whether the user's state is determined to be a sleeping,resting, running, working out, or other activity level dependent state,as only examples. The user's state is determined by a controller of theearphone system based on at least two different types of sensor data,such as measured first motion data and measured bio-signal type data.For example, the detected first motion data and detected bio-signal typedata may be respectively captured by one or more first motion sensorsand one or more bio-signal sensors included in one or more earpieces ofthe earphones. The user's state may be further determined based onadditional measured data, such as based on captured audio type data,measured cadence or stride type data, and/or determined/measuredlocation type data. The user's state may also be determined based on anadditional detected second motion data, such as by a second motionsensor arranged in the earphone system distinct from the earphones ofthe earphone system. Still further, either or both of the exampledetected first or second motion data may be further considered or usedto filter the detected bio-signal, such as to remove motion artifacts,e.g., prior to consideration of the bio-signal type data with the, forexample, first motion data to determine the state of the user. Acontroller of the earphone system, such as included in either of theearpieces or a remote control unit in communication with the earpieces,may then selectively provide or adjust audio output to the usercorresponding to the user's determined state.

In the following, while earphones may be described in relation to in-earearphones, embodiments equally include on-ear earphones, headphones, andor a single earphone in, for example, a Bluetooth headset for a phone,any wired or wireless communication configuration from a mobile orstationary home audio equipment, as well as one or more wirelessheadsets and audio distribution system to be used during guided tours,theme parks, public gyms, as only non-limiting examples. Embodimentsherein may additionally include examples where the earphone(s) areconfigured to interact with one or more central audio equipmentavailable for multiple connections to earphones.

In addition, herein “audio output” should be understood as referring toany sound content that may be transmitted or generated from an audiosource to a speaker unit of the earphones. The audio output includes butis not limited to music, relaxation sounds, speaking, pod casts, audiobooks etc. Moreover, herein “media player” should be understood as beingany source from which audio output may be generated and/or transmitted,including but not limited to a smart phone, music player, DVD player,radio, TV, computer, tablet etc. Thus, herein, a media player herein mayrefer to a portable or mobile processing device and/or a stationaryprocessing device, which can either or both generate and transmit audio.The media player may obtain the corresponding audio data or read theaudio data from an internal memory of the media player. Alternatively,the controller of one or more of the earpieces or a remote control unitof the earphone system may be configured to perform the operations ofthe media player 13.

FIG. 1 illustrates a side view of an earphone according to one or moreembodiments. Referring to FIG. 1, an earphone 1 may include an earpiece2 that includes a front side portion 3 and an active side portion 4, forexample. The earpiece 2 may further include a rim 5 arranged between thefront side portion 3 and active side portion 4, for example.

The front side portion 3 and the active side portion 4 may be physicallyarranged to oppose each other, for example, and may joined together bythe example rim 5.

In an example, the earpiece 2 may further include a recess 6. Forexample, the recess 6 may be configured so as to create a slot orcrimping section across a portion of the earpiece 2. The recess 6 may bearranged on any of the front side portion 3, active side portion 4, andrim 5. For example, the recess 6 may be formed only on the front sideportion 3, formed on the front side portion 3 and rim 5, formed on thefront side portion 3, the rim 5, and the active side portion 4, formedon the rim 5 and the active side portion 4, or formed only on the activeside portion 4. When formed on the front side portion 3 or active sideportion 4 the recess 6 may be formed near the rim 5, though embodimentsare not limited thereto. In an example, a cable 7 may be removablelyarranged in the recess 6. For example, the recess 6 may stationarilyhold or grip the cable 7, such as through friction or a crimping orgripping force of the recess 6.

The active side portion 4 of the earphone is configured to be insertedin connection to an auditory canal in a user's ear 20 in order to enablethe user to listen to audio output transmitted via the earphones. Forexample, the active side portion 4 may be slightly tapered for enablinga close fit to the user's auditory canal. The active side portion 4 maybe formed by a soft and partially resilient material so that theearpiece 2 provides a comfortable fit in a user's ear, and to preventrubbing. The active side portion 4 and the front side portion 3 may bedifferent materials, or they may be the same materials. In an example,the front side portion 3 may be of a polymer material, steel, or othermaterial, as only examples. For example, it may be an advantage if thematerial of the front side portion 3 is a hard or firm material, and/ora wear resistant material, e.g., so as to protect the components insidethe earpiece 2.

The front side portion 3 and/or the active side portion 4 may beconfigured so the earphone may be entirely or mostly retained inside theouter ear, such as to not extend outside of the ear, which may make theearphone suitable for use in a wide range of activities or states of theuser. For example, in the example where the earphone is configured to beentirely or mostly retained inside the outer ear, the earpiece 2 may notdisturb sleeping even if the user is resting on the side of their head,such as with the ear being supported by a pillow or the like, since theearpiece 2 does not, or does not mostly, extend outside of the outerear.

FIG. 2 illustrates a front view of an earphone according to one or moreembodiments. As illustrated in FIG. 2, the cable 7 may be attached tothe earpiece 2 at a first end 10. For example, when the cable 7 isconfigured to include a communication interface or wired cablingconnecting the earpiece 2 to a remote control unit and/or microphone, ora remote control unit, microphone, and/or media player when the cable 7is alternatively or additionally wiredly connected to a media playercompared to a wireless embodiment, the first end 10 may include orrepresent an orifice or opening in the rim 5, the active side portion 4,and/or the front side portion 3 to provide further communication betweenthe internal communication interface or wired cabling of the cable 7 tointernal components of the earpiece 2. Alternatively, the first end 10may be representative of a connection unit or interface where the cable7 terminates and through which electrically connects to the internalcomponents of the earpiece 2. The recess 6 may be configured or arrangedat or along a second end 11 of the earpiece 2, such as illustrated inFIG. 2 in physical arrangement with the front side portion 3 and rim 5.As noted above, the recess 6 at or along the second end 11 may beconfigured or arranged on or in the front side portion 3, the activeside portion 4, and/or the rim 5. In an example, such as illustrated inthe example of FIG. 2, the second end 11 may be a portion of theearpiece 2 physically arranged on an opposite portion of the earpiece 2at the first end 10. For example, by placing an entrance position of thecable 7 at the first end 10 at the first side portion 3 and rim 5 of theearpiece 2, and the recess 6 on the example opposite side of theearpiece 2 at the second end 11, the loop may be adjustable orexpandable to have a shape as similar to the curve of the concha of theear of the user as possible.

The recess 6 may be arranged either on an outer portion of the earpiece2, such as the example second end 11, or more centrally, such as morecentrally on the front side portion 3, as only an example. The recess 6may extend in a direction in line with, at an angle to, orperpendicularly relative to the entrance of the earpiece 2 at theexample first end 10. Also, in an example, the front side portion 3 mayhave a convex exterior surface, e.g., the surface of the front sideportion that is away from the ear, and may be configured to, when inuse, be substantially level or even within a plane represented by theuser's outer ear surface, such as the auricle, rim, helix, or pinna ofthe ear. Thereby, any sharp scrubbing edges of the earpiece 2 may beavoided which may be uncomfortable for the user. Further such aconfiguration may allow a user to rest the ear 20 against an object,such as a pillow, without the earpiece 2 obstructing the comfort for theuser.

FIG. 2 also illustrates an example where the cable 7 is arranged in therecess 6 along the length of the cable 7. This example may provide aslidable arrangement of the cable 7 in the recess 6. For example, byarranging the cable 7 in the recess 6, an adjustable loop 9 may beformed in the cable 7 between the first end 10 and the second end 11. Inan example, the recess 6 may stationarily hold, grip, or fix the cable 7so the loop 9 may be fixedly or stiffly maintained, such as so the loop9 may be fixedly placed in or relative to the concha of a user's ear tomaintain a position of the earpiece 2 relative to the ear. For example,the recess 6 may be configured so the cable 7 is slidably arranged inthe recess 6, such as to provide the adjustable or expandable loop 9arranged to run along the curve of the concha in the ear 20 in order toprovide stability for the earphone 1 to help prevent the earpiece 2 fromaccidently falling out of the ear 20. Here, though cable 7 is describedas connecting internal cabling of the cable 7 to internal components ofthe earpiece 2 through the opening of the earpiece 2 at the first end10, the cable 7 is not required to connect to internal components of theearpiece 2, and thus, the opening of the earpiece 2 at the first end 10may not be required. Rather, the first end 10 may alternatively hold orgrip the cable 7, such as through another recess similar to the recess6, to provide the slideable or expandable loop 9 for maintaining theposition of the earpiece 2 relative to the ear and in cooperation withrecess 6.

For example, FIGS. 3-4 illustrate a front view of an earphone in useaccording to one or more embodiments, where the loop 9 is illustrated asbeing placed in the concha 21 of the user's ear.

Here, depending on embodiment and the variability of configured holdingforce of the recess 6, the loop 9 may be adjustable in order to allow apressing action against the ear 20 when pressing against the concha 21in order to retain the earpiece 2 inside the ear 20 of the user. FIGS. 3and 4 illustrate separate examples of how the size of the loop 9 may bevariously adjusted to have a tailored fit for the user's ear 20, such asby variously sliding the cable 7 in the recess 6. Thereby the loop sizemay be adapted to the user's ear and thereby have a tailored fit. Forexample, FIG. 3 demonstrates that the loop 9 with an adjusted shorter orsmaller size may fit ear 20 differently from a longer or greater sizemay fit ear 20 in FIG. 4, so the user may be able to variously adjustthe loop size to have a tailored fit to maintain the earpiece 2 withinthe user's ear 20. For example, the loop 9 may thereby hold or fix theearpiece 2 inside the ear 20 and prevent the earpiece 2 from falling outduring, for example, sleeping, running, dancing etc. etc.

As an alternative, the cable 7 may be configured to merely connect tointernal components of the earpiece 2 through either of the examplefirst end 10 or second end 11, or other portion of the earpiece 2, andthe loop 9 may be a separate cable or stiff element or frame configuredbetween the non-limiting example first end 10 and second end 11.

FIG. 5 illustrates a perspective view of an earphone according to one ormore embodiments. Referring to FIG. 5, the active side portion 4 isshown protruding from the earpiece 2, such as in the aforementionedexample configurations for insertion into the user's ear canal.

Here, though examples have been discussed regarding the configuration ofthe active side portion 4 being an in-ear active side portion 4,embodiments are not limited thereto. For example, the active sideportion 4 may be an out of ear active side portion 4, to sit or restoutside of the user's ear canal. For example, when the active sideportion 4 is configured to sit or rest outside of the user's ear canal,a side pressure from a pillow, for example, acting on the earphone whileresting the head against the pillow may not apply pressure on thesensitive ear canal, thus providing an earphone further adapted forsleeping. Also, as noted above, in an example the earpiece 2 isconfigured to be retained inside the outer ear and not extend outside ofthe ear, which may make the earpiece 2 further suitable for use in awide range of activities or states of the user. For example, if theearpiece 2 is retained inside the outer ear and/or the active sideportion 4 is an out of ear active side portion, the earpiece 2 may notdisturb sleeping even if the user is resting the ear on a support like apillow or the like. As an alternative, the active side portion 4 may beinterchangeable between the in-ear and out of ear configurations, and/orfor respectively different sizes for fitting for various user's ears.

FIG. 6 illustrates an earphone system according to one or moreembodiments. Referring to FIG. 6, the earphone system includes earpieces2, with respective loops 9, cable 7, a remote control unit 12, and mediaplayer 13. The remote control unit 12 may be arranged along or in-linewith the cable 7 of the earphones. In this example, the remote controlunit 12 is connected via a cable 7 to control operations of the mediaplayer 13, such as for selective control for the audio output of themedia player 13 or for performing other non-audio output relatedoperations of the media player 13. The media player 13 may also be anycomputing device that is configured to provide or stream audio eitherwirelessly and/or through one or more wires, such as cable 7. Forexample, the media player 13 may be a mobile phone, wearable smartdevice, tablet, exercise equipment, television, stereo receiver, etc.Thus, the remote control unit 12 may connect with either or both of theearpieces 2 through cable 7 and/or wirelessly, such as in an alternateexample where the media player 13 communicates with the earpiece 2 andthe remote control unit 12 through a wireless communication protocol,such as Bluetooth, WiFi 802.11 compliant network, or other communicationprotocol, including ad hoc networks.

In addition, in an example, the remote control unit 12 is wirelesslyconnected to either or both of the earpieces 2 and the media player 13,while the earpiece 2 is connected through cable 7 to the media player13. For example, the remote control unit 12 may be selectively attachedthe cable 7 or operable separate from the cable 7 when connected to, orincluding, a corresponding power source.

FIG. 7 illustrates a flow chart describing an earphone method accordingto one or more embodiments, such as implemented by the above earphonesystem of FIG. 6 and/or below discussed earphone system of FIG. 8, asonly examples. Thus, though the operations of FIG. 7 will be explainedthrough reference to similar reference numbered components as in FIG. 6or FIG. 8, and thereby imparts the availability of such exampleoperations on the earphone systems of FIGS. 6 and 8, or any of theremaining previous discussions, this is for explanatory purposes, andthus embodiments are not limited thereto.

Referring to FIG. 7, one or more earphones may be configured in anearphone system that includes a remote control unit 12 attached on acable 7 used to connect one or more earphones to the media player 13. Inaddition, while some earphones may communicate wirelessly with either orboth of corresponding remote control units 12 and the media player 13,other earphones in the earphone system may communicate throughcorresponding cables 7 with either or both of corresponding remotecontrol units 12 and the media player 13. Alternatively, in an example,the remote control unit 12 and/or either or both of the earpieces 2 ofan example earphone may communicate via a wireless protocol with themedia player 13. The wired and wireless communications in these settingsmay include wiredly and/or wirelessly preparing/transmitting/receivingany or any combination of control data to/from one or more of theearpieces 2 of corresponding earphones, to/from corresponding remotecontrol units 12, and/or to/from the media player 13 to control therespective audio output of any of the one or more earpieces of one ormore earphones in the earphone system dependent on respectivelydetermined states of the corresponding listeners. For example, one ormore of the earphones may communicate the control data wirelessly withthe media player 13 and one or more corresponding remote control units12, and the one or more corresponding remote control units 12 maycommunicate control data through one or more example cables 7 with themedia player 13, or one or more of the earphones may communicate controldata through the one or more example cables 7 with the media player 13and one or more corresponding remote control units 12, and the one ormore corresponding remote control units 12 may communicate the controldata wirelessly with the media player 13. As another example, the one ormore earphones may communicate control data wirelessly with the mediaplayer 13, and the one or more earphones may communicate through one ormore cables 7 with the one or more corresponding remote control units12, which may communicate through one or more cables 7 with the mediaplayer 13, or the one or more earphones may communicate control datathrough one or more corresponding cables 7 with the media player 13, andthe one or more earphones may communicate wirelessly with the one ormore corresponding remote control units 12, which may communicatewirelessly with the media player 13. Still further examples areavailable, such as an all wired environment, an all wirelessenvironment, there being multiple media players in the earphone system.

In these examples, the wired or wireless communication control data mayalso include sensor data, e.g., collected by one or more of earpieces 2of the one or more earphones or the corresponding remote control units12, to be analyzed by the one or more earpieces 2, the correspondingremote control units 12, the media player 13, and/or one or more remoteservers remotely connected to the media player 13, to determine a stateof a respective user, the determination of which can subsequently berelied upon by any of the one or more earpieces 2 of the one or moreearphones, corresponding remote control units 12, or media player 13 tocontrol the output of the audio output for that respective user.

As only an example, the control data may also be control data to controla selecting or implementing of specific tracks, playlists, genres,BPM-tracks, adjusting volume, pausing, stopping, playing, etc., such asbased on the determined state of the user and/or indicatingpredetermined, or user preset, corresponding controls that will beimplemented for various determinable states of the user. Based on suchexamples, FIG. 7 illustrates operations for adjusting an audio outputfrom the media player 13 by one or more earpieces 2.

In this example, and with further reference to the example earphonesystem of FIG. 8, one or more of the earpieces 2 may each include aspeaker element 24 for converting an audio output into sound waves, acontroller 120, a first motion sensor 25, and pulse rate detector, e.g.,including an example IR led 26 and IR sensor 27, noting that additionalor alternative motion and bio-sensor detectors may be available.

In FIG. 7, a detecting operation 201 is performed by the controller 120to control the motion sensor 25, and thereby detect movements of a userbased on movement detections by the first motion sensor 25, which isconfigured to generate sensor data representative of or indicatingmovement or changes in movement. A pulse rate detecting operation 202 isalso performed by the controller 120, where the controller 120 controlsthe detecting of a pulse rate through control of the pulse ratedetector, which is configured to generate pulse rate sensor dataindicating a measured pulse rate of the user or other bio-signal data.As illustrated in FIG. 7, operations 201 and 202 may be performed inparallel, noting that embodiments are not limited thereto.

In operation 203, the respective sensor data generated in operations 201and 202 may be transmitted to the controller 120. For example, thecontroller 120 may control the motion sensor 25 and the pulse ratedetector to transmit their respectively measured sensor data to thecontroller 120.

In operation 204, a state of the user is determined by the controller120, for example. The state of the user may be determined by thecontroller 120 based on the received sensor data from the motion sensor25 and the received sensor data from the pulse rate detector.

In operation 205, based on a result of operation 204 with respect to thedetermined state of the user, the controller 120 may control selectionor adjustment of audio playback or audio output of the media player 13,for example. In an example, the controller 120 may control the mediaplayer 13 to select or adjust audio output for provision to theearpieces 2, such as through the aforementioned wired or wirelesstransmission of control data from the controller 120 to the media player13, and thereby control the media player to provide the selected oradjusted audio output to the speaker elements of the earpieces 2. If thecontroller 120 is performing operations of the media player 13, then thecontroller selects or adjusts audio output and provision of the selectedor adjusted audio output to the speaker elements of the earpieces 2.

Accordingly, operation of the media player 13 and the earpieces 2, forexample, may be controlled by the controller 120 based on considerationof both measured movement data and measured bio-sensor data of the useras collected by of one or more of the earpieces 2.

FIG. 8 illustrates an earphone system according to one or moreembodiments. The earphone system of FIG. 8 may correspond to theearphone system of FIG. 6, and implement the operations of FIG. 7,though embodiments are not limited thereto. Thus, while components ofFIG. 8 may be explained through reference to similar reference numberedcomponents as in any of the above discussions, and thereby imparts theavailability of such example operations or components on the earphonesystem of FIG. 8, this is for explanatory purposes, and thus embodimentsare not limited thereto.

With respect to FIG. 8, the earphone system includes an example earpiece2, energy harvesting component 30, and controller 120, for example,which may be implemented in a wired, wireless, or combination of wiredand wireless environments. The earpiece 2 includes a speaker element 24for playback of audio output, a motion sensor 25, and a pulse ratedetector. In an example, the motion sensor 25 is an accelerometer. Thepulse rate detector may include an IR led 26 and an IR sensor 27, asonly an example. The pulse rate detector may alternatively be anotherbio-signal detector.

In an example, the motion sensor 25 is an accelerometer adapted to sensemotions of the user. For example, the accelerometer may be a 3Daccelerometer. By allowing the motion sensor to be an accelerometer, itmay be possible to instantly detect a change in the motion pattern of auser, as well as monitor trends in motion. A user may for instance at acertain time be walking, and at a time later start jogging. Theaccelerometer detects accelerations, i.e. changes in movements. Theacceleration or movement pattern would be different between when theuser walks, jogs, or sprints, for example. As the accelerometer detectssuch changes in motion, the example controller 120 may receive thecorresponding sensor data from the motion sensor 25, determine the stateof the user, and control the earphones according to a predeterminedsetting. In an example, one or more motion sensors may be located in oneor both of the earpieces 2 of an example earphone. These motion sensorsmay be referred to as first motion sensors, for example. Thereby,through measurements of these one or more first motion sensors, motionsof the users head may be registered.

Further, the IR sensor 27 may sense the reflected light from the IR led26, and from that reflection, the pulse rate detector may detect andcalculate the blood flow and consequently the pulse rate. Alternatively,the sensed data of the reflected light by the IR sensor 27 may betransmitted to the controller 120, for example, where the controller 120calculates the blood flow and consequentially the pulse rate. In anexample, the IR led 26 uses a wavelength of about 800-900 nm, e.g. about850 nm. In an example, each earpiece 2 may include pulse rate detectors,or one or both of the earpieces 2 may include multiple pulse ratedetectors, which may provide improved accuracy in determining the stateof the user.

By using the IR sensor 27, it may be possible to detect the blood flowwithout having an external pulse rate detector device in closeconnection to the body, such as strapped or taped. The IR sensor 27 maybe placed in or on one or more of the earpieces 2 at a place to beclosely positioned to a user's body. For example, the IR sensor may beplaced in or on the active side portion 4 of the earpiece 2 that isbeing inserted into the ear or being held close to the ear. By placingthe IR sensor in or on the earpiece 2, the blood flow may be detectedfrom a spot in the ear. IR sensors may further be easy to use and maynot require any further installation on the body. Here, though such anIR sensor pulse rate detector has been discussed as an availableexample, embodiments are not limited thereto. For example, a conductivesensor material for detecting the electrical conductivity of the user'sskin may additionally or alternatively be used to determine the pulserate of the user.

In an example, in addition to using sensor data from the motion sensor25, i.e., referred hereinafter as the first motion sensor 25, fordetermining the state of the user, because the first motion sensor 25may detect minor relative motions between the earpiece 2 and the ear,the sensor data from the IR sensor 27 may be filtered so as to avoiddisturbance sensor data by filtering out motion artifacts thatcorrespond to the detected relative motions between the earpiece 2 andthe ear of the user as detected by the first motion sensor 25.

In addition, a second motion sensor, e.g., a second accelerometer, maybe implemented in a remote control unit 12, such as the remote controlunit 12 of FIG. 6, and may be provided for further improved accuracy indetermining the state of the user, e.g., in addition to the use of thefirst motion sensor 25. A second motion sensor may additionally oralternatively be implemented in the media player 13, with any or anycombination of the first and second motion sensor data being used todetermine the state of the user. In an example, only one earpiece 2 mayinclude the first motion sensor 25 and the example pulse rate detector,or only one of the earpieces 2 includes the first motion sensor 25 andthe other earpiece 2 includes the pulse rate detector, or when bothearpieces 2 include both motion and heart rate sensors only onesensor/detector may be selectively operated in each earpiece 2. In anexample, with respect to the motion sensor 25 in the earpiece 2, by onlyhaving sensors in one earphone the costs may also be kept lower.Moreover, in an example, sensor data transmitted from the IR-sensor maybe analog data or in an analog signal form, though embodiments includeeither detector or sensor data to be transmitted to the controller 120as analog signals and/or digital signals.

Here, in the example where the earphone system further includes theexample remote control unit 12 of FIG. 6, the controller 120 may beincluded one or both of the earpieces 2 that are separate and at adistance from the remote control unit 12. The remote control unit 12 maycommunicate with the earpieces 2 through the example cable 7 of FIG. 6or wirelessly. Alternatively, the remote control unit 12 may include thecontroller 120 and the second motion sensor. Though examples areprovided where the second motion sensor is included in the remotecontrol unit 12 or the media player 13, embodiments are not limitedthereto. In FIG. 8, the remote control unit 12 may be represented bycontrol buttons 123, for example.

The user may use the remote control unit 12 to manually control theaudio output, or alternatively the user may use the earpieces 2 tomanually control the audio output. Moreover, the second motion sensormay be used in combination with the first motion sensor 25 to determinemotions of the user with even higher accuracy. Further, by having twomotion sensors, sensor data from both first and second motion sensorsmay be used in the determining of the state of the user, or the sensordata from the first motion sensor 25 may be used for filtering thedetected pulse rate, e.g., so as to avoid disturbance or artifact sensordata resulting from relative motions between the earpiece 2 and the earof the user, and the sensor data of the second motion sensor may be usedin combination with the filtered heart rate data to determine the stateof the user. In an example, the second motion sensor may be a secondaccelerometer.

Moreover, the illustrated controller 120 may be a micro controller 120that is communicatively connected to one or more of the earpieces 2. Inexamples, the controller 120 may be an included component of one or moreof the earpieces 2. A control of the adjustment or selection of audiooutput by the controller 120 may be automatically or autonomouslyperformed by the controller 120. In another example, one or more of thesensor data from the first motion sensor 25 and pulse rate detector maybe offloaded to the media player 13, for example, and one or more of theoperations to determine the state of the user and the providing,adjusting, and/or selective playing of audio output through the speakersof the earpieces 2 may be performed in a distributed manner. Thereby, inan example where the controller is at least one of the earpieces 2 orthe remote control unit 12, some of the calculations that are performedto determine the state of the user and/or the providing, adjusting,and/or selective playing of audio output may be distributed between thecontroller 120 integrated in the earphones and the media player 13having a higher calculation capacity, or all such calculations may beperformed by the media player 13.

In an example, communication between the earpieces 2 may be implementedthrough the cable 7, such as illustrated in FIG. 6. The cable 7 mayinclude a signal cable dedicated for digital communication, as well asaudio signal cables. For example, a dedicated cable for the sensor datamay be used. Alternatively, as noted above, any of the communication maybe implemented wirelessly.

As noted above, the controller 120 may be, or arranged in, theaforementioned example remote control unit 12 of FIG. 6, for example,and/or configured in either or both of the earpieces 2. The controller120 may be a microcontroller, for example. The controller 120 and theremote control unit 12, such as when the remote control unit 12 and thecontroller 120 are separate components of the earphone system, may eachinclude an I²C circuit 121 configured to respectively providecommunication with other components of the earphone system, such asbetween the controller 120 and the example remote control buttons 123and/or from the pulse rate detector, e.g., either or both of sensors 26and 27, and the first motion sensor 25. Moreover, the controller 120 andthe remote control unit 12 may each include a wireless communicationmodule, which is hardware communication circuitry, such as a Bluetoothcircuit for wireless communication, e.g. a Bluetooth low energy (BLE)122 or 802.11 Wifi circuit or other standardized wireless communicationprotocol, such as a protocol that has such lower power consumptioncapabilities, configured to provide wireless communication to/from themedia player 13, or other devices, as discussed above. The communicationmodule, or another hardware communication module may be configured toalso implement cellular or other telecommunication protocols, such as3G, 4G, 4G LTE, 5G, etc. In an example, the controller 120 may beincluded in one or more of the earpieces 2 and the remote control unit12 may be arranged on the example cable 7 connecting the earpieces 2 andthe media player 13 when the earpieces 2 connect wiredly to the examplemedia player 13, for example. When the earpieces 2 connect wirelesslywith the media player 13, the remote control unit 12 may connectwirelessly with either or both of the earpieces 2 and the media player13 or wiredly with the earpieces 2 and wirelessly with the media player13. Alternatively, the remote control unit 12 may be configured toperform the operations of the media player 13. Still further, in anexample, the illustrated controller 120, energy harvesting component 30,power control 28, microphone 35, and phone connection 33 may beconfigured in one or both of the earpieces 2, or distributed between theearpieces 2 of an example earphone system. In such an example, theremote control unit 12 may be physically separate from the earpieces 2,and either of the earpieces 2 or the remote control unit 12 may performthe operations of the media player 13. Still further, in such anexample, the phone connection 33 of the earpieces 2 and/or a likecomponent of the remote control unit 12 may include phone communicationhardware, such as for making and receiving phone calls or data. Stillfurther, in another example, the phone connection 33 of FIG. 8 isrepresentative of a mobile processing device, such as a smart phone ortablet having a user interface, such as for controlling selecting ofuser determinable states and corresponding associated audio controls oradjustments discussed herein, and one or both of the earpieces 2, e.g.,the controller 120 in at least one of the earpieces 2, and the remotecontrol unit 12 may be configured to communicate with the phoneconnection 33.

Accordingly, the controller 120 may determine, based on respectivemotion and heart rate sensor data, that the user is falling asleep, andaccordingly then control a volume of audio output to be lowered, aspecific song or genre to be selected, or a playlist pre-associated withsleeping be activated, e.g., in order to not risk that the user may wakeup abruptly due to high noise. In another example, if the controller 120determines based on the respective motion and heart rate sensor datathat the user is starting or beginning to work out, the volume may beincreased, a different genre selected, or a playlist pre-associated withworking out may be activated. More specifically, the sensor data fromthe respective sensors may be analyzed, a result of which differentiatesbetween different activities of the user, e.g., between running andlifting weights at the gym, and the controller 120 can then adjust theaudio output accordingly.

Here, the user may preset such playlists that are to be associated withmultiple different determinable states, such as through a user interfaceof the media player 13 or another device user interface, such as a smartphone, that is in communication with the controller 120. Thereby, theuser may select music or other audio which is suitable for the detectedstate of the user, in addition to the availability to adjust volume,pause or stop the audio etc., in response to the determination of thestate or state change. In addition, how audio output is adjusted fordiffering states or changes in states may be implemented by thecontroller 120 according to a user pre-defined setting associated withthe determined state of the user. Thus, the user may self choose how theaudio output is to be changed depending on differing determinable statesof the user, as well as the example self choosing of types of music,genre, or playlists. The settings may be handled through the exampleuser interface of the media player 13 or of the other device, such as aconnected smart phone. The control of the audio output may be predefinedby the user for any specific state that a user may find appropriate. Forinstance, a user may upload to the media player 13, or identify thelocation or name of the same, a playlist of music, a genre, or an audiobook that may be suitable when in a resting state, or choose to pausethe audio or decrease the volume if the controller 120 determines thatthe state of the user is the falling asleep state. Another playlist ofaudio may be defined to be played when a work out state is determined bythe controller 120. Alternatively, or as a complement, the user maypredefine a variety of sound content which beats per minute correspondto different user states, and the controller may control the audiooutput accordingly. In an example, the user may also at any timemanually override the determined state-based control of the media player13, e.g., if a certain audio output may not be desirable at a giventime, such as through controls of the example remote control unit 12 orthe touch or tap controls of the earpiece 2, for example.

In addition, in an example where the audio output is music andcontrolled based on predetermined association with the detected state ofthe user, the controller 120 may further perform a matching of themusic's beat per minute (bpm) with the detected activity by the firstmotion sensor 25. For example, the controller 120 may control the audiooutput to be a selected music that has a related beat to the state ofthe user, such as a song with fast pace (high BPM) being controlled tobe played when the user is determined to be running at a high speed(e.g., based on a detected high heart rate) during a determined runningstate (e.g., based on the detected high enough heart rate and detectedsufficient motion), while a song with a slower pace (slower BPM) may becontrolled to be played when the state is determined to be a slowerrunning or jogging (e.g., based on a detected lower heart rate) duringthe determined running state. The controller may thereby choose musicaccording to its properties, such as beats per minute, frequency andsuch in order to match a user's state with the right music that may besuitable for the corresponding activity or state. Thus, in examples,audio output may be based on the determined state of the user dependenton both the detected pulse rate and the detected motion, and further themusic's beat per minute (bpm) may be adjusted to correspond to thedetected pulse rate. By adapting the controller 120 to be able toperform such a matching between a user's detected pulse and a piece ofmusic's beat per minute, it may facilitate for the user to be provided apiece of music most suitable or most desirable to the user's state.Moreover, the matching of the music's beat per minute (bpm) may be basedon a detected cadence of the user. The cadence may be, for example, thenumber of revolutions of a crankset of a bicycle per minute when thedetermined state is bicycling, or a step frequency of the user when thedetermined state is walking or running. For instance, if the user isbicycling, the controller 120 may obtain cadence sensor data from asensor integrated in or attached to the bicycle, or other machine, todetermine the cadence for the matching. In addition, if the user isrunning, and periodically starts sprinting, the change in pulse/cadencemay also be detected and the controller 120 may receive the changedmotion data and adjust the audio output to a music that hassubstantially similar or at least related beats per minute as the user'snew or alternating pulse/cadence. Thereby the user does not have tomanually adjust the music, which is desirable to avoid during a workoutsince any manual adjustment may be time-consuming and taking focus fromthe physical activity.

In another example, if a user is awake and walking, the respectivesensor data from the heart rate detector and the first motion sensor 25may in combination provide sufficient data to the controller 120 for thecontroller 120 to discern a correct state of the user and control aprovision of correct audio output and/or adjustment of already providedaudio. If such a user subsequently sits down, the pulse rate detectormay generate sensor data that indicates a slower pulse and the firstmotion sensor 25 may generate sensor data that indicates a motionpattern associated with sitting, which may be sufficient data for thecontroller 120 to determine that the state of the user is resting, andto control a change in audio output, such as by controlling a decreasingin the volume of the audio output or a switch in the audio output to alow beat (beats per minute) music audio output. Still further, if thecontroller 120 determines that the state of the user thereafter changesto the user beginning to fall asleep, or having fallen asleep, whilelistening to audio output, the control 120 may further lower the volumeof the audio output or alter the audio output, and/or may then furthercease the audio output based on the detected changed state of the user.

Thus, in an example, the determination of the state of the user includesdetermining whether the user is asleep or awake, which may still furtherinclude determining whether the user is falling asleep from an awakestate and whether the user is waking from a sleep state. For example, bycombining data of a measured pulse rate and sensed motion of the user,asleep, if the pulse is determined to have slowed down to a rest pulseand the controller 120 further determines that there has not beensufficient change in motion over time, the controller 120 may determinethat the user is in a sleep state. For example, the controller 120 maydetermine that the detected pulse rate is a resting pulse rate and thata detected motion sensor sensing corresponds a moving patternpre-associated with sleeping, and be able to determine that the user isin the asleep state. Here, the example moving pattern that is associatedwith sleeping may be adjusted over time as the user uses the earphones.In other examples, the moving pattern that is associated with sleepingmay correspond to aggregated detected movements over time being below athreshold value. In another example, the controller 120 may control theaudio output to be gradually adjusted when the determined state is thatthe user is falling asleep, such as by gradually decreasing the volumeduring a determined time span where the user is gradually fallingasleep. In an example a resting pulse rate may also be predefined by theuser and/or the resting pulse rate may be automatically determined andadjusted over time by the controller 120 and or machine learningimplemented by the controller 120 or the media player 13.

In another example, the controller 120 may further determine whether thestate of the user is the resting state or whether the user is in aworking out state.

Thus, the controller 120 may control the audio output to be adapted tothe determined activity state of the user. For instance, the controller120 may determine that there is a combination of a detected low pulserate and a detected slow or minimal motion, so the controller 120 maydetermine that the user is in the rest state. Here, for example, therest state may be distinguished from an asleep state in that thedetected pulse rates and/or the detected moving patterns are differentbetween the rest state and the asleep state. Moreover, a user being atrest may be more probable to desire a different audio output than whenthe same user is working out. For example, the controller 120 maydetermine that the state of the user is a working out state, e.g.,including the controller 120 determining between plural working outsub-states of running, lifting weights, bicycling, climbing, dancing,jumping, doing yoga, stretching, etc. Each of the sub-states of theworkout state may be determined based on different detected heart rateranges, for example, within detected heart rates that are interpretableto be the workout state. The controller determinable working outstate(s) may also be defined by the user in a user interface of anotherdevice, e.g., a smart phone, that is connected to the controller 120 ora user interface of the media player 13. Location information may alsobe generated or obtained by the earphones, the controller 120, or themedia player 13, such that when the user is determined to be visiting agym, the controller 120 may automatically control the audio output toprovide “bicycle race” by Queen at high volume when the determinedsub-state is a warming up a spinning bicycle, to provide “Eye of thetiger” by Survivor while when the determined sub-state is a benchpressing state, to provide “I will survive” by Gloria Gaynor when thedetermined sub-state is a burpees exercise state, and/or to provide “Nowoman no Cry” with Bob Marley at a low volume to the user when thedetermined sub-state is a stretching state. Thus, based on thisprogression, for example, the controller 120 may also consider preset orlearned progressions of such different exercises, or activitysub-states, to determine the activity state of the user, such as for theexample gym location. In these examples, the different audio or volumesmay be based on predetermined, e.g., by the user or by a manufacturer,preferences of the user, such as the user being more likely to preferhigher volume when running compared to when stretching after the run.

In an example, sensor data from either or both of the first motionsensor 25 and the second motion sensor may indicate any change in auser's movements and the sensor data from the pulse rate detector may beused to determine in what state on a sleep-to-workout-scale the user isin. By combining the sensor data from the first motion sensor 25, forexample, and the sensory data from the pulse rate detector, informationabout a user's state may be determined with higher accuracy thanprevious attempts that based the user's activity only on a single typeof sensor data, for example.

Herein, a “state of the user” is to be broadly understood as referringto a user's physical state or activity. It includes, but is not limitedto, user states like the user being awake, resting, sleeping, workingout, running, lifting weights, bicycling, climbing, dancing, jumping,etc. Further, the state of the user also includes transition states ofgoing from awake to falling asleep, such as gradually falling asleep,and consequently based on such a determined transition state thecontroller 120 may lower the volume of the audio output gradually.Moreover, the different detectable or discernable states of the user mayalso be predefined by the user, including adding to or deleting from anypreset states that may have been predetermined by a manufacturer, suchas through a user interface of the media player 13 or through a userinterface of an application on another device, such as a smart phone,that is in communication with the controller 120. Thereby, if the userwants the earphones to be able to detect a specific state, say dancing,the user can define the state dancing and corresponding respectivedancing state indicating motion and heart rate sensor data. Moreover,the controller 120 may further include computer readable code or have amachine learning configuration that learns the user's different statesand corresponding respective sensor data. Thereby, the controller 120can further improve its ability and accuracy to determine the state ofthe user.

Moreover, a power control unit 28 may be included in the earphone systemfor controlling when the different components are to be activated,deactivated, and inactive. The power control unit 28 may also beelectrically coupled to the energy harvesting component 30 including anenergy storage circuit 31 and a boost converter circuit 32, for example.The energy storage circuit 31 may include a micro capacitor, thoughembodiments are not limited thereto. As only an example, the energyharvesting component 30 may include trickle charge circuitry configuredto harvest energy from the phone connection 33, such as an example wherethe energy harvesting component 30 harvests energy from a 3.5 mm plug ata terminating end of the cable 7 where energy may be harvested from theplayed audio provided through cable 7 or a microphone line included inthe cable 7. Here, this discussion of the energy harvesting component 30is an example of implementation of the energy harvesting component 30 ina wired environment, while embodiments are not limited thereto asadditional and/or alternative embodiments are also available. Forexample, the energy harvesting component 30 may harvest energy from anyother type of phone connections or transmissions of the phone connection33, non-phone connections or transmission, or any other type of radiatedor transmitted energy. With the energy harvesting component 30 nointernal or external battery may be needed, or smaller sized batteriesmay alternatively be available, e.g., compared to a system without suchan energy harvesting capability, and the energy harvested by the energyharvesting component may be used to boost the internal battery reservesof the controller 120, such as where the controller 120 is included inone or more of the earpieces 2, which enables a more compact design. Forexample, with the energy harvesting component 30, no exterior chargingor replacing of batteries may be needed for the measuring of motion orpulse rates. For example, by having a local energy storage circuit 31and a boost converter circuit 32 the motion sensors(s) and the pulserate detector(s) may be selectively operated or driven with enoughenergy during active periods, e.g., without an external power source.For example, the energy harvesting component 30 may be controlled toharvest energy during inactive periods, such as when the user isdetermined to be not listening to audio or not wearing the earpiece 2,to store energy for a next active period when the user is wearing theearpiece 2, for example. As another example, with the energy harvestingcomponent 30, less energy may be needed from the media player 13 in thewired environment example when the earpieces 2 are wiredly connected tothe media player 13. In addition, with the energy harvesting component30, the sensors may be active in longer sequences with the same amountof battery usage.

In addition, in an example where the earpieces 2 are wirelesslyconnected to the media player 13, and no cable is needed, the user mayhave increased flexibility relative the media player 13. However, insuch an example, the earpieces 2 may require a separate battery to drivethe respective speaker elements and the first motion sensor 25 and pulserate detector. Thus, in an example, the pulse rate detector may becontrolled to be active for select periods and inactive for periods, soas to save energy even while the user is using the earphones to listento audio or when the user is not using the earphones. Thereby, lessenergy may be needed compared to when continuously measuring the pulserate. The pulse rate detector may be the most energy consuming componentof the earpieces 2, and thus, it may be desirable to reduce energyconsumption of the pulse rate detector. The pulse rate detector may becontrolled, e.g., by the controller 120, to have inactive periodslasting for between 1 second and 10 minutes. In some examples theinactive periods lasts for five seconds, ten seconds, 30 seconds, 1minute, 5 minutes or at any other period of time depending onembodiment. Further, the active period for measuring the pulse may becontrolled, e.g., by the controller 120, to be between 1 second to 1minute. In some examples the active periods lasts for one second, threeseconds, ten seconds, 30 seconds, 45 seconds, or any other active periodof time that may provide sufficient data. Thus, in this example, energymay be saved and thus the battery of the earpiece 2 and/or the batteryof the example media player 13 may last longer.

The first motion sensor 25 (and/or the second motion sensor) may also becontrolled, e.g., by the controller 120, to be inactive for selectperiods of time. The inactive periods of the motion sensor 25 may lastfor between 5 second and 5 minutes, for example. In an example, theinactive periods may last for five seconds, ten seconds, 30 seconds, or1 minute. Correspondingly, the controller 120 may control the activeperiods of the first motion sensor 25 to last for between 5 second and 5minutes, for example. In some examples the active periods lasts for fiveseconds, ten seconds, 30 seconds, or 1 minute.

In one example, the first motion sensor 25 (and/or the second motionsensor) in the earpieces 2 may be further used for handling manualcommands of a user. For example, the controller 120 may monitor themotion sensor, for example, for indications of input commands and beconfigured to selectively control the audio output when the first motionsensor 25 of an earpiece 2 detects such a tap or touch command on theearpiece 2, the controller 120 may determine what the input tap or touchis a command intended to control the audio output, such as controls topause/play through a single tap, to change a song, playlist, or genrethrough a double tap, and to activate a voice control through a tripletap, etc., only examples. In an example, the state determiningactivity/operations of the controller 120 may be momentarily disabledfor a period of time when a manual tap action is detected, so as to notincorporate sensed motion data corresponding to the tap into thedetermination of the state of the user.

Moreover, the earphone system may include the microphone 35, e.g., inone or both earpieces 2 and/or the remote control unit 12, so that theearphone system may be used to record a sound from the user, e.g., forhandling a phone call or voice commands for a connected media player 13,such as where the media player 13 is a smart phone. Moreover, themicrophone 35 may be used to further increase the reliability of thedetected state of the user. For example, if the user state is determinedby the controller 120 to be a running state (i.e., based on therespective sensor data), the microphone may further be used forconfirming that captured environmental sounds correlate to predeterminedsounds associated with running. In another example, if the user state isdetermined by the controller 120 to be a sleeping state (i.e., based onthe respective sensor data), captured sound by the microphone may beused for confirming that the captured environmental sounds correlate topredetermined sounds associated with sleeping. In still furtherexamples, it the controller 120 may control the microphone to captureambient sounds and based on that the captured ambient sounds thecontroller 120 may control the volume of the audio output to beincreased. For example, in a determined noisy environment the volume maybe adjusted to be higher than in a determined silent environment. Themicrophone 35 may alternatively or additionally be configured with inthe media player 13, and the controller 120 may consider all capturedambient audio when determining the activity of the user.

Moreover, the controller 120 or the remote control unit 12 may include auser selectable physical on/off switch for enabling or disablingoperations of the controller 120, for example, in controlling the audiooutput based on the user's detected state, e.g., selectively controllingwhether operations of FIG. 7 are implemented. Accordingly, the user canprevent the audio output from being changed through user control of thecontroller 120 when the user changes activity by turning the on/offswitch off, if the user so desires. In other embodiments, the on/offswitch may be implemented as a user interface of an application on themedia player 13, a user interface of the controller 120, of the remotecontrol unit 12, or of another device in communication with thecontroller 120 or remote control unit 12, for example. As still anotherexample, the example on/off switch may be a touch sensitive switch for atouch interface of the earpiece 2.

In an example, one or more or all operations described above may beimplemented by one or more processors configured in the earpiece 2, theremote control unit 12, or the media player 13 such as through by acomputer readable code stored on/in one or more non-transitory computerreadable media of the example earpiece 2, remote control unit 12, ormedia player 13, that when executed by the one or more processors, causethe one or more processors to perform the one or more or all suchoperations. For example, the non-transitory computer readable media maystore computer readable code to cause a controller to adjust the audiooutput from a media player 13 to earphones according to one or more orall embodiments described above. The non-transitory computer readablemedia may store computer readable code to cause the example one or moreprocessors to detect a state of a user through sensor data from twodifferent types of sensors, to detect corresponding movements of theuser by control one or more motion sensors, to detect a pulse rate ofthe user through sensor data from one or more bio-signal sensors, suchas one or more pulse rate detectors, and to control the one or morebio-signal sensors, and/or to control a transmitting of such respectivesensor data from the one or more motion sensors and the one or morebio-signal sensors to the one or more processors. Further thenon-transitory computer readable media may store computer readable codeto control a selecting or an adjusting of a providing to, or receipt of,audio output, e.g., from such a media player 13, output through aspeaker of the earphones, e.g., based on the determined state of theuser.

The earphones 1, earpieces 2, media player 13, remote control unit 12,first motion sensor(s) 25, second motion sensor(s), pulse ratedetector(s), IR led 26, IR sensor 27, speaker 24, accelerometers,controller 120, I²C circuit 121, MCU, BLE circuit 122, control buttons123, power control 28, energy harvesting component 30, energy storage31, boost converter 32, microphone 35, phone connection 33,communication module, and remote server, as only examples, describedwith respect to FIGS. 1-8 and that perform the operations described inthis application are implemented by hardware components configured toperform the operations described in this application that are performedby the hardware components. Examples of hardware components that may beused to perform the operations described in this application whereappropriate include controllers, sensors, generators, drivers, memories,comparators, arithmetic logic units, adders, subtractors, multipliers,dividers, integrators, and any other electronic components configured toperform the operations described in this application. In other examples,one or more of the hardware components that perform the operationsdescribed in this application are implemented by computing hardware, forexample, by one or more processors or computers. A processor or computermay be implemented by one or more processing elements, such as an arrayof logic gates, a controller and an arithmetic logic unit, a digitalsignal processor, a microcomputer, a programmable logic controller, afield-programmable gate array, a programmable logic array, amicroprocessor, or any other device or combination of devices that isconfigured to respond to and execute instructions in a defined manner toachieve a desired result. In one example, a processor or computerincludes, or is connected to, one or more memories storing instructionsor software that are executed by the processor or computer. Hardwarecomponents implemented by a processor or computer may executeinstructions or software, such as an operating system (OS) and one ormore software applications that run on the OS, to perform the operationsdescribed in this application. The hardware components may also access,manipulate, process, create, and store data in response to execution ofthe instructions or software. For simplicity, the singular term“processor” or “computer” may be used in the description of the examplesdescribed in this application, but in other examples multiple processorsor computers may be used, or a processor or computer may includemultiple processing elements, or multiple types of processing elements,or both. For example, a single hardware component or two or morehardware components may be implemented by a single processor, or two ormore processors, or a processor and a controller. One or more hardwarecomponents may be implemented by one or more processors, or a processorand a controller, and one or more other hardware components may beimplemented by one or more other processors, or another processor andanother controller. One or more processors, or a processor and acontroller, may implement a single hardware component, or two or morehardware components. A hardware component may have any one or more ofdifferent processing configurations, examples of which include a singleprocessor, independent processors, parallel processors,single-instruction single-data (SISD) multiprocessing,single-instruction multiple-data (SIMD) multiprocessing,multiple-instruction single-data (MISD) multiprocessing, andmultiple-instruction multiple-data (MIMD) multiprocessing.

The methods illustrated in FIGS. 1-8 that perform the operationsdescribed in this application are performed by computing hardware, forexample, by one or more processors or computers, implemented asdescribed above executing instructions or software to perform theoperations described in this application that are performed by themethods. For example, a single operation or two or more operations maybe performed by a single processor, or two or more processors, or aprocessor and a controller. One or more operations may be performed byone or more processors, or a processor and a controller, and one or moreother operations may be performed by one or more other processors, oranother processor and another controller. One or more processors, or aprocessor and a controller, may perform a single operation, or two ormore operations.

Instructions or software to control computing hardware, for example, oneor more processors or computers, to implement the hardware componentsand perform the methods as described above may be written as computerprograms, code segments, instructions or any combination thereof, forindividually or collectively instructing or configuring the one or moreprocessors or computers to operate as a machine or special-purposecomputer to perform the operations that are performed by the hardwarecomponents and the methods as described above. In one example, theinstructions or software include machine code that is directly executedby the one or more processors or computers, such as machine codeproduced by a compiler. In another example, the instructions or softwareincludes higher-level code that is executed by the one or moreprocessors or computer using an interpreter. The instructions orsoftware may be written using any programming language based on theblock diagrams and the flow charts illustrated in the drawings and thecorresponding descriptions in the specification, which disclosealgorithms for performing the operations that are performed by thehardware components and the methods as described above.

The instructions or software to control computing hardware, for example,one or more processors or computers, to implement the hardwarecomponents and perform the methods as described above, and anyassociated data, data files, and data structures, may be recorded,stored, or fixed in or on one or more non-transitory computer-readablestorage media. Examples of a non-transitory computer-readable storagemedium include read-only memory (ROM), random-access memory (RAM), flashmemory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs,DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, magnetictapes, floppy disks, magneto-optical data storage devices, optical datastorage devices, hard disks, solid-state disks, and any other devicethat is configured to store the instructions or software and anyassociated data, data files, and data structures in a non-transitorymanner and provide the instructions or software and any associated data,data files, and data structures to one or more processors or computersso that the one or more processors or computers can execute theinstructions. In one example, the instructions or software and anyassociated data, data files, and data structures are distributed overnetwork-coupled computer systems so that the instructions and softwareand any associated data, data files, and data structures are stored,accessed, and executed in a distributed fashion by the one or moreprocessors or computers.

While this disclosure includes specific examples, it will be apparentafter an understanding of the disclosure of this application thatvarious changes in form and details may be made in these exampleswithout departing from the spirit and scope of the claims and theirequivalents. The examples described herein are to be considered in adescriptive sense only, and not for purposes of limitation. Descriptionsof features or aspects in each example are to be considered as beingapplicable to similar features or aspects in other examples. Suitableresults may be achieved if the described techniques are performed in adifferent order, and/or if components in a described system,architecture, device, or circuit are combined in a different manner,and/or replaced or supplemented by other components or theirequivalents. Therefore, the scope of the disclosure is defined not bythe detailed description, but by the claims and their equivalents, andall variations within the scope of the claims and their equivalents areto be construed as being included in the disclosure.

What is claimed is:
 1. An earphone apparatus, the apparatus comprising:an earpiece, including a speaker, configured for arrangement relative toa user's ear for listening to audio from the speaker; a bio-sensor; amotion sensor; and a controller configured to determine an activitystate of the user based on aggregated sensor data from the bio-sensorand the motion sensor over time, and to control audio output to the userbased on the determined activity state.
 2. An earphone method of anearphone apparatus having earpieces configured for respectivearrangements relative to a user's ears for listening to audio, themethod comprising: controlling respective measuring of bio-signals andmotion changes of a user by the earpieces; aggregating sensor data ofthe measured bio-signals and motion changes of the user over time;determine an activity state of the user based on the aggregated sensordata; and selectively controlling the audio output to the user based onthe determined activity state.